METHOD OF DETECTING FLICKER , AND VIDEO CAMERA 
USING THE METHOD 



Field of the Invention 

The present invention relates to a method of 
detecting lighting induced flicker in a video signal, 
and to a video camera equipped for carrying out this 
5 method. 



Background of the Invention 

Artificial lighting derived from alternating 
current sources, particularly fluorescent lighting, 
contains a strong brightness modulation component, or 

10 flicker, at twice the frequency of the alternating 
current sources. This factor of 2 arises from the 
power relation between instantaneous voltage of the 
alternating current sources and instantaneous 
brightness, and from the trigonometric relation cos 2 

15 (x = 0 . 5 (1+cos (2x) ) . Commonly encountered flicker 
frequencies are 100 Hz in Europe and 120 Hz in the 
United States. Although invisible to the human eye, 
flicker may be highly visible to image sensors. The 
problem is most apparent at low exposure values. An 

20 imagine sensor samples this modulation waveform as 

reflected from objects in the scene and reproduces it 
perfectly. 
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Solid-state sensors fall into two broad 
categories according to exposure method. One category 
- is full-field, where all pixel elements of the sensor 
are exposed simultaneously. A second category is 
5 rolling window", where all pixel elements in a sensor 
row are exposed simultaneously, but the onset of 
exposure is delayed from row to row. Lighting flicker 
induces a periodic variation in luminance, known as 
banding. Banding is apparent in the time domain, and 

10 in the case of rolling-window sensors banding is also 
apparent in the vertical spatial domain. 

In the case of the rolling-window sensors, 
should the camera and the frequency of the alternating 
current source be in perfect synchronization, the 

15 modulation pattern will be temporally frozen, appearing 
as static luminance banding down the image. However, 
the problem is compounded if camera field rates and 
frequency of the alternating current source differs by 
some amount, causing the luminance modulation bands to 

2 0 roll up or down the image. The rate of roll depends 

mostly on whether the camera is operating home or away, 
i.e., nominal frame rate may be a close sub-multiple of 
the frequency of the alternating current source. For 
example, a 50 Hz camera operating in the United States 

25 is operating away. Roll associated with a camera 
operating at home is extremely slow, while roll 
associated with a camera operating away is much faster. 

As well as being visibly distracting to the 
viewer, luminance modulation generates considerable 

30 frame-to-frame differences in image streams which 
could, for example, make the difference between a 
software video CODEC performing acceptably. Thus, it 
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is important that a camera system be capable of 
detecting and cancelling artificial lighting flicker. 

Detection of lighting flicker in the spatial 
domain is difficult in the case of rolling-window 
5 exposure sensors, and is much more difficult in the 
case of full-field exposure sensors. In the former 
case the difficulty is due to potential strong 
correlations between expected banding patterns caused 
by lighting flicker and variations in actual scene 
10 luminance. 

U.S. Patent 5,053,871 discloses a still video 
camera which uses a previewing technique to provide 
automatic exposure control and flicker detection. 
However, there is a need to provide flicker detection 
15 in motion video cameras. U.S. Patent 5,272,539 

discloses a video camera with flicker detection, but in 
this prior arrangement the detector frame rate is 
coupled with the video frame rate, which limits its 
usefulness . 



20 Summary of the Invention 

An object of the present invention is to 
provide a time-domain technique for detecting and 
reducing the frequency of flicker for motion video 
cameras, and which is also capable of being applied to 
25 both full-field exposure sensors and rolling-window 
exposure sensors. 



Brief Description of the Drawings 

An embodiment of the invention will now be 
described, by way of example only, with reference to 
30 the drawings, in which: 



FIG. 1 is a schematic representation of a 
photosensitive array according to one embodiment of the 
present invention; 

FIG. 2 illustrates a sampling method 
5 according to the present invention; 

FIG. 3 is a block diagram of the flicker 
detection method according to the present invention; 
and 

FIG. 4 is a block diagram showing use of the 
10 method according to the present invention in a video 
camera. 

Detailed Description of the Preferred Embodiments 

Referring to FIG. 1, a photosensitive array 
comprises a main array of pixels 10. It will be 

15 appreciated that FIG. 1 is highly schematic, with only 
a small number of pixels 10 being shown. Additionally, 
the photosensitive array comprises one or more (in this 
embodiment, two) super-pixels 12 and 14. Each of the 
super-pixels 12, 14 differ from the pixels 10 of the 

20 main array in two principal ways 

The super-pixels 12, 14 are physically large 
in comparison to the pixels 10 of the main array so 
that they may stand a better chance of imaging some 
part of the scene which contains a flickering light 

25 source or reflects such a flickering source. In this 
example, each super-pixel is one entire column of 
photosensitive pixel elements 10 which have been 
electrically connected in common. 

The super-pixels 12, 14 are exposed and 

30 sensed in a manner independent from the pixels 10 of 
the main array. While each line of the main array is 
sensed at the frame rate dictated by each application, 
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each super-pixel 12, 14 is sensed independently, 
usually at a rate much higher than the sensor frame 
rate to produce a suitable sequence of readings in each 
period of the lighting flicker. A convenient rate at 
5 which to sense each super-pixel 12, 14 is the line-rate 
of the application, which is usually some hundreds of 
times faster than the frame-rate. 

Separate means must be provided to control 
the gain of each super-pixel 12, 14 to ensure its 
10 output sample falls within its linear operating range 
while maximizing dynamic range. As stated above, each 
super-pixel 12, 14 may be provided by connecting in 
common a column of standard size pixels, as indicated 
by interconnection line 20 in FIG. 1. 
15 The output of each super-pixel 12, 14 is then 

operated on by a detection mechanism which will now be 
described with reference to FIGS. 2 and 3. The 
following description refers to the use of a single 
super-pixel. The detection mechanism operates ad 
20 infinitum on successive length-N sequences f (n) of 

compound samples. Each compound sample comprises one 
or more accumulated individual samples s(a)of the 
super-pixel. Each compound sample is spaced apart by 
an appropriate interval I, with the interval I being 
25 referred to as the compound sampling interval. 

The individual super-pixel samples s(a) are 
accumulated over a fixed number of lines A, less than 
or equal to the interval I, and is .referred to as the 
compound sampling aperture. Such accumulation allows 
30 an ensemble reduction of random components contained in 
each super-pixel reading s(a) at the expense of 
amplitude reduction of the super-pixel signal at the 
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frequencies of interest. This is attributable to the 
roll-off effect of a sampling aperture: 



f (n)=-£s(a) 

A a =l 

Note that in the cases where the desired 
5 compound sampling interval I cannot be expressed as an 
integer multiple of the sensor line interval, the 
compound sampling interval can be adjusted on an 
instantaneous basis to average out to the desired 
interval over time. The resultant phase jitter is 

10 tolerable as long as the compound sampling aperture 

remains constant. FIG. 2 illustrates the composition 
of the sequence f (n) for N=3 . 

One example of a detection mechanism takes 
the form of a bandpass filter tuned to the nominal 

15 frequency of the flicker. If the compound sample rate 
of the super-pixel is chosen as a multiple of the 
nominal flicker frequency, a straightforward detector 
might use the fundamental output component F(l) of a 
radix-N butterfly, or N-rotor. This circuit performs 

20 complex correlations with the fundamental Nth-root of 
unity to produce the instantaneous measure of complex 
flicker energy E: 

E = F (1) = Ef (n) e 2 * N 

While radix-2 is the simplest butterfly, its 
25 response is phase-dependent and therefore unreliable. 



As N increases, so does hardware complexity, and the 
smaller the compound sampling interval and potential 
aperture. We have found that N = 3 or 4 yields the 
most efficient and effective results. 
5 These instantaneous complex flicker energy 

readings E must be averaged over time in some manner to 
produce a longer term estimate E' of flicker energy. 
One example of an averaging mechanism is the "first- 
order autoregressive filter, or leaky integrator, whose 
10 ability to track phase drift may be traded against 
noise immunity by its system time constant ]l, and 
updating long term average E r with an instantaneous 
measure E: 

E' = Eu + E' (1 - u) 

15 The process of magnitude extraction affords 

some protection against phase drift, an inevitable 
consequence of short term or long term differences 
between actual and nominal flicker frequencies. The 
final flicker detection decision should be based on the 

2 0 magnitude or modulus of long term average E' . For 
example, if T is some programmable or predefined 
threshold, then the Boolean decision variable d can be 
defined as follows: 

d = | E' I > T 

25 Note that the compound sampling interval may 

be chosen to undersample the flicker signal, relying on 
the folding or aliasing effect to detect harmonics of a 
notional sub-harmonic of flicker. While this method 
allows longer exposure times or compound sampling 
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apertures, it is less able to track flicker frequencies 
differing from the nominal. This is so since the error 
in an instantaneous angular frequency is greater than 
that of the fundamental case for a given difference 
5 between actual and nominal flicker frequencies. 

FIG. 4 shows the foregoing method used in a 
flicker detecting video camera. The main sensor array 
10' has its exposure setting controlled by either the 
output of an automatic exposure control circuit 18 of a 

10 known type, or by a flicker free exposure setting. The 
choice between these two is controlled by the Boolean 
operator and as derived above. 

The actual correction of lighting flicker, 
once detected and identified in frequency, is 

15 relatively straightforward. To expand on the sampling 
analogy, it is well known that increasing a sampling 
aperture away from the therotical perfect sampling 
(i.e., convolution with a dirac-delta pulse train) 
causes a roll-off of the frequency response which obeys 

20 the equally well known sin(x)/x or sine function. If 
the exposure window is considered as a sampling 
aperture, then those temporal frequencies present in 
the scene whose period coincides with the temporal 
duration of the exposure window, or harmonics of such 

25 frequencies, will be rendered invisible, as they 

coincide with nulls in the sine function. Setting the 
exposure period to the inverse of a suspected flicker 
frequency or one of its harmonics will then provide 
effective banding removal. 

30 A weakness of this scheme can arise under 

bright lighting conditions. Here the sine function 
approaches the origin and no sine function null can be 
found which corresponds to a desirable exposure 
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setting. Without recourse to additional exposure 
control mechanisms such as LCD shutter or mechanical 
iris, a compromise must be sought between acceptable 
banding and acceptable exposure setting. The invention 
5 thus provides a technique for detection and frequency 
identification of flicker which operates in the time 
domain, and which is applicable to both full-field 
exposure sensors and to rolling-window exposure 
sensors . 

10 Modifications and improvements may be made to 

the foregoing embodiment within the scope of the 
invention. 



